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ABSTRACT 

Research vas undertaken to develop a system for 
predicting completion time in a self-paced training course. The 
hypotheses were developed that: 1) course content-related instruments 
would be better predictors of completion time than general aptitude 
measures; and that 2) a linear predictive function would provide the 
best description of the relationship between predictor variables and 
completion time criteria. A predictive test battery which consisted 
primarily of content-related tests and mini-lessens that were 
directly associated with a training course^ proved to be better than 
general aptitude measures for predicting course completion time; a 
strong linear relationship was found between predictors and 
coEpletion time criteria. Step-wise linear regression analyses were 
used to develop predictive functions and to determine potentially 
useful predictors. Computer programs and nomographs were constructed 
to display the predictive equations and were recommended for use. 
Since there were significant inter-group differences in learning 
rates depending on the media used^ the development of media-specific 
predictive equations was recommended. (Author/PE) 
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FOREWORD 



The purpose of the research performed as part of HumRRO Work Unit STOCK was 
to develop practical techniques for the management of entry-MOS training programs, in 
order that these programs might more effectively use individualized instruction for 
students of all aptitude levels. The training management techniques developed under 
Work Unit STOCK were evaluated and refined as part of Work Unit PRISM. 

This document reports on the design, development, and initial evaluation of an 
apprqach to predict ''time-to-learn'* in a student self-paced training program. A previous 
report, Self-Paced Advanced Individual Training (AIT) and Duty Assignment Procedures 
(Technical Report 73-14) identified the need for a procedure that accurately predicts 
course completion time in order for individualized instruction to be successfully 
implemented. 

Research performed under Work Units STOCK and PRISM was conducted by 
HumRRO Division No. 1, Alexandria,' Virginia, Dr. J. Daniel Lyons, Division Director. 
The Work Unit Leader for the research was Dr. C. Dennis Fink; Dr. Harold Wagner was 
Work Sub-Unit Leader and was directly responsible for conduct q^f the research. Dr. Richard 
D. Behringer designed and developed the Predictive Test Bal^^tery used in this study. 
Dr. Currell L. Pattie administered the testing program and collected the data required 
for analysis. The location of the research was the U.S. Army Quartermaster School, 
Fort Lee, Virginia. 

Enlisted men assigned to the project during the period covered in this report were 
SP4 Edward T, Weston who developed the PREDICT com.puter program, and SP4 Darrell 
L. Anderson who constructed the nomographs included , in this report. Both men con- 
tributed to the data processing and programming activities required during the research. 
. / HumRRO research for the Department of the Army undex Work Unit STOCK-PRISM 
is /conducted, under contract DAHC 19-73-C-0004. Aj-my Trajning Research is performed 
urider Army Project 2Q062107A745. 



Meredith P. Crawford 
President 

Human Resources Research Organization 



OBJECTIVES 



The pur|)osi> ()[ tlu' r(»scarrh piTformed as part of Work^Ufnt STOCK was to develop 
practicai tcchiiiiiiu's for manaj^in^ trainin^{ pro-ams, so that individualized instruction 
could he used more cffintively . The Iraininj^ manai^ement techniques developed under 
Work I'nit S'lXX.'K w^^rc evaluated and refined as part of Work Unit PRISM. 

PROBLEM 

A Auida mental [)r()bleni to he solved in the impk^mentation of self-paced training is 
how to a('(uirat('ly predict each student's course 'completion date prior to graduation. 
Tfiesc prcdictioi. arc- neccied so tiiat trraduates can he assigned to their duty positions or 
to additional training in a timely manner. This proV:>lem of accurately predictinj^ i^elf- 
paced course completion time was the focus of the effort described in this report. 

APPROACH 

'I'lie course that served as tht^ rt^sciircli vt^hicle for this study was the Stock Control 
and Accounting Sp.ecialist (MOS 7(>P20) course. Hiis course was bein^ individualized at 
tile D.S. Army C^iarrt^rn}astei- School ((JMS). 

A literature suit(\v in the art^a of "time-to-l(\irn" predictions led to the follow. 
assumptions; 

(1) Cc)urse cont(Mit-r(dateti instruments ar(* tietter predictors of completion time 
thaii general aptitude^ measures. 

(2) A linear predictive function is the ''best'' description of the relationship 
l)etvveen pretiictor variables and completion time criteria. 

Th(^se assumptions formtni the basis of the approach taken, which involved the use 
of standardiz(\l aptitude^ tests and course-related instruments as predictors of completion 
time. Th(^ pr(Hlictiv(^ test batteries that were developed contained three different types of 
t(\sts. First, ihv.y included ''aptitude-like'' tests of those skills and knowledges (e.g., 
arithmetic, vocabulary) considered necessary to enable the student to acquire the MOS* 
related behaviors. The sec:ond type of test can be described as a **mini-lesson/test'* 
situation, hi e.ss(Mic(\ these wer(^ samples of the ta>-ks to be encountered later in the 
course. .\ small amount of instruction in each task was given to the students, who were 
then tested on what they had just learned to do. The third type of test instrument 
consisted of scales designed to tap the "affective" (or motivational) factors considered 
im[)ortant in the subject niatt(»r area covered by this course. Scores on these tests, which 
were administered prior to entry in the cours(\ were used with the standardized test 
scores in the development of completion time predictions. Predictive equations were 
generated by the use of the st(^p-wise multiple regression analyses described in the report. 

As the entire course had not been completely individualized at the time of this 
study, the criteria reflected only that portion of the course which had been self-paced. 
The criterion time scores were ol^tained by instructors of the 76P20 course at the 
Quartermaster School. The MumRRO staff member stationed at Fort Lee organized and 
forwarded the dala to Alexandria, where it was' proc(*ssed and analyzed. The completion 
time data were usinl to determine the accuracy of the predictions. 



FINDINGS AND IMPLICATIONS 

The rosearch findings showed that scores oi)tained h'om lIumRRO-doveloped tests 
correlated more strongly with completion time criteria tlian did the standardized ACB 
scores. However, the AFQT score was a significant component of several useful predictive 
equations. The assumption that course content-related predictors would be beU^M* th^m 
{general aptitude measures for estimating completion time was supported by these 
findings. • 

Stepwise multiple linear regression analyses were employed in the development of 
predictive functions and the determination of potentially useful predictors. Some multiple 
correlations obtained between the predictive variables and completion time criteria ranged 
from .65 to .75, while others ranged from .85 to .87. The latter correlations occurred 
with the use of within -course time scores as predictors in the etjuation, wh&reas the first 
s(-»t of corrplations resulted from the use of only the pre-course test scores. Therefore, tiie 
second assum.ption, that a strong linear relationship exists between predictors and com- 
pletion time criteria, was considered tenable on the basis of these findings. 

A computer program was developed for displaying the predictive information. This 
pro^^r^m was considered to l)e a useful aid to managers of self-pa('ed training programs, 
hut was limited in that it required access to a computer facility. Nomographs were 
constructed to display the predictive equations in a more generally useful manner. The 
nomographs were simple to use and did not require access to a computer. They were 
recommended for use in the prediction system at the QMS. 

In order to estimate completion time accurately, the data had to be grouped 
according to the instructional niode employed. There were significant differences in the 
rate of lefirning between groups trained using different media. Therefore, media-specific 
predictive equations needed to developed. This led to the hypothesis that if the 
predictive instruments and the sltUGiion in which they are administered are more clos(My 
related to the instructional situation, [predictions will be improv(^d. This hyf^othesis has 
yet to be tested. 
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chapter 1 
INTRODUCTION 



THE PROBLEM 

The recent trend toward individualized training in the Army has been accompanied 
by a need to solve the administrative problems thai are generated by such training. It is 
the characteristic of student self-pacing that produces many of these management 
problems^ In the Army, self-pacing has produced at least two types of problems: 

(1) How to efficiently utilize "early'' graduates at the local training base, as 
they may be constrained from overseas assignments by Public Law 51.' 

(2) How to insure the timely arrival of assignment instructions, so that the 
students may depart their training base immediately upDn graduation. 

These problems were identified in Work Unit STOCK (1), a project which i.ad as its 
ol^jective the successful implementation of an individualized self-paced course at the U.S. 
Atmy Quartermaster School (QMS). This work began with a survey of self-paced . training 
programs in all ^ the Services. The survey was followed by a thorough analysis of the 
Army*s personnel assignment system and its effects on self-paced training programs. Both 
the military-wide survey and the Army assignment system analysis resulted in identical 
conclusions— there, is a military-wide need to accurately predict each student's course 
completion date prior to graduation, in order for self-paced training to be successfully 
implemented. 

The problem of accurately predicting self-paced course completion time is the focus 
of the effort described in this report. A course undergoing individualization at the QMS 
(Stock Control and Accounting Specialist— MOS 76P20) served as the research vehicle for 
this study. In addition to being self-paced, the 76P20 course uses a *'multi-media" 
approach to present the instruction. In Work Unit STOCK, a technique was devised to 
predict MOS 76P20 course completion time. The procedures and instruments that make 
up this technique were evaluated and refined in Work Unit PRISM. This report describes 
the development and current status of that effort. 

RELATED STUDIES OF "TIME TO LEARN" 

The use of '*time to learn" as a critical variable in recent education and training 
research st^ms, in part, from a model developed by John" B. Carroll and described by 
Block (2). Carroll's ''Model of School Learning** was essentially a conceptual paradigm 
that outlined various factors influencing a student*s success in school learning. Carroll 
found that a student's aptitude fjredicted not only the level to which he learned in a, 
giuen time, but also the amount of time he required to learn to a gioen level Rather than 
viewing aptitudes as determiners of what level of learning a student could reach, Carroll 
defined aptitudes as determiners of the amount of time required to learn a task to a 
given criterion level under ''ideal'* instructional conditions. 

* PL 51 constrains AJT grarluates from de Tting for overseas assignments until the completion of 
eight weeks formal training or on-the-job training (OJT), beyond the eight weeks used in Basic Combat 
Training. 



it was B!oom (3J who transformed this conceptual model into an effective 
working model for ''mastery*' learning. In describing the model, Bloom stated that if 
students are normally distributed with respect to aptitude, and the kind and quality of 
iiisiruction and the amount of time available for learning are made appropriate to the 
rtiararteristics and needs of each student, then the majority of students may He 
(»:-:pe< led to achieve mastery of the subject. Thus, the Carroll model and Bloom's 
i xtonsion of it support the notion that, given an unlimited time to learn (and if other 
factors been controlled for or optimized), mastery will be obtained by each student 

at a different rate. Bloom {3) stated, 

**We believe that the student should be allowed the time he 
nci^ds to learn a })articular subject. I.,earning time needed will be 
affected by his aptitudes, his ability to understand the instruction, 
^h'ui the quality of instruction he receives in the class and outside of 
class. An effective mastery learning strategy must find ways of 
altering the time individuals need for learning as well as providing the 
time necessary for each student. Such a strategy, therefore, must 
solve the instructional as well as the school organizational (including 
time) problem/' 

Recently Atkinson describing criteria for a theory of instruction, devoted a 

.suhsUintial j:>ortion of his model to the fcU'tor of ''time to learn." Although it is only one 
of the ingrefiients in his instructional mociel, time is c'onsidered by Atkinson to be one of 
iliv irn{>ortant factors to vary when determining the optimal instructional strategy for 
eacf) student. 

Iruiividual student rate-to-criterion has been suggested as a basis for evaluating 
indivuiualized instruction (5), The basic assumption for this suggestion is that there are 
stat)le [)erformance rates for students within the context of an ideal instructional system, 
i Ikjs. l)y ke(\[)ing achievement constant, measuring the time required to attain the 
objectives can be employed in a framework for evaluating individualized instruction. Such 
a nieasurenuMit ind(\\ was utilized to evaluate the efficiency of a self-paced economics 
course (o). • 

I{esearch concerning the "time to learn" factor has been performed in situations 
wliere individualized instruction is contemplated, or in operation. For example, at the 
Tniversity of Pittsburgh there has been a concentrated effort to develop individualized 
jr.rtruction (i.e., Individually Prescribed Instruction - IPI). In this context there have been 
several investigations of learning rate (7,8,2)- Several methods of measuring rate of 
learning were studied. The authors found that the rate of learning was not constant over 
various learning tasks, and that aptitude and other general predictor variables did not 
rstiinat(^ these It^aniing rates adequately. 

Muml^RO rest'archers have attempted to predict completion time's for self-paced, 
[irogramnHHi inslru( tion courses, but only general aptitude scores were available for use as 
predictor vjiriables and unsatisfactory predictions resulted.^ Stankard (10) attended to the 
time factor wh(Mi implementing computer-assisted instruction. Faced with the scheduling 
and managerial problems that accompany self-paced training, attempts were rhade to find 
])redict()rs of student learning rate within the CAI ^situation. These attempts were 
unsu(HH\ssful. A move mstrucfional sitiiation-related approach toward prediction was 
suggested. 

According to ('arroll (ii), the findings described above could be due to a learning 
rale tiial is quite specifir to a partieular task. He stated that the best way of estimating 
time to us<' tests of ''relevant aptitudes and learnings." This opinion is supported by 

' An .'Xpiurjiory .malvMN of" compU'iion times at vari(>us Army training centers of pro^^rammeci 
Q Mi- • ! .ii ! K .!! {ri.ii r I uils for scvi'r.il MOSs w as prrformed at Division No. i in June 11)72. 



some Oi the results obtained in liumKRO Work Unit IMPACT (12). First, it was shown 
that different kinds of learning tasks recjuire specific and unicjuo student abih"ties— that 
'"general" tests are inadequate predictors of studt^nt performance, it was also determined 
that some measures of personal cluu'acteristics were useful in predictint^ student per- 
formance early in the course, luit were not the saine as those that predicted performance 
in later, more complex stages. 1'hese findinj^s support the view that early instruct ioruU 
experience's within the training program may overcome son'u* of the personal backgrcnmd 
factors brought into the situation, and te.ese in-cours(^ factors would then be the most 
'Mmportant" determiners of performance within the coun-;c\ 

The studies eitiKl contain much evidence, both direct and indirect, lo support the 
supposition that specific task-related ])redictors of iearnijig rate are better than jnost 
general aptitude measures. 

The studies in this area have used a lineiir model to predict learning tiiue or 
performance. In doing so, many studies have relied on multiple linear regression analyses 
for the pur[)ose of predicting performance (13, hi). In the study by Owen and Feldhusen 
(14), the most useful prediction model was a stepwise multiple regression approach that 
was based on the assumption that . . .variables which are similar both in history and 
composition should predict one another better than variables further removed in time and 
different in structure." It was hypothesized that the best predictors of achievemeiU wer(^ 
obtiiined from performance in a Himilar enuirotitncnt. This hypothesis was supported in a 
study by Wardrop (15) in which miniature learning situations vjei'v used to j)redict 
classroom performance and proved to be better predictors than ""general" measures. 

In HumRRO Work Unit FRKDICT, multiple regression analyses were used for th(^ 
purpose of predicting performance. The objective of that study was to develop systems 
for predicting aviator success in training and in operational assignments (16, 17). In a 
study by Dees (18) a psychomotor skill, a measure of perseverance, and a measure of 
leadership ability were selected as criteria to l)e predicted by a battery of 37 tests 
employed in a stepwise multiple regression analysis approach. A much greater predictive* 
capability was found than would be obtaintnl from **general iiUelligen'ce tests'' alone. 

The studies cit(»d are only a small sample of tlu^ studi<\s that have used multiple 
regression analyses to predict perforrnanct\/achievement. They, too, support the use of 
specific, content-related predictor variables. Bas(»d upon tins line of evidence. Work Unit 
STOCK resciirchers assumed the following: 

(1) Course conte^^nt-rejati'd tests are better predictors uf completion time than 
gtnieral aptitude measures. 

(2) .A linear predictivi' function is the *M)est" description of the rclationshij) 
between [)redict()r variables and completion timt- criteria. 

OBJECTIVES OF THE STUDY 

The primary obj(\ tive of this researcli effort was to develop a ttH'hnicjut^ that would 
provide accurate course completion time prtniictions. Tliis technic jue was based ui)on th.e 
two assumptions described abcA'e. Thus, this study indirectly assessed the validity .'of 
those assumptions. / 

A secondary objective was to [irovide. for a self-))aced course (MOS 7{iP20), a 
system that woukl (a} predict individual training times for its participants, and (b) display 
these predictions in a manner that would hv usef\d to training managers who admin isU^r 
the individualized progi^am. 



The iterative approach used to develop completion time predictions is outlined 
below. For explanatory purposes, the events wiil be described in four separate phases, 
although some of the activities occurred as overlapping steps. These four phases can be 
described as follows: 

1. Exploratory Stage 

n. Development of Predictive Test Battery 1 

III. Development of Predictive Test Battery 2 

IV. Proposed Study Using Predictive Test Battery 3 



Chapter 2 



PHASE I ^ EXPLORATORY STAGE 



COLLECTION OF AVAILABLE "PREDICTIVE" INFORiVlATION 

The major goal of this Phase was to determine the ''least usefuT' predictor variables, 
ill order to eliminate them from further consideration: To accomplish this, all infer- 
mation that might be of use for^ predicting course completion time was collected from 
76F20 course trainees. This Included all the scores on standardized aptitude tests that 
were recorded in the trainees' personnel files. Although other attempts at predicting 
completion time using these "general'* aptitude measures had been unsuccessful (10), this 
was the only information about each trainee, that could be obtained at the beginning of 
Phase I. Also, it was not known if these ^'general" measures would be useful for 
predicting learning rate in the Supply content area. 

The Armed P'orces Qualification Test (AFQT) and the tests comprising the Army 
Classification Battery (ACB) were the major variables investigated in this Phase. These 
tests are given under a variety of conditions. The ACB is a set of 11 instruments 
administered at the reception station during the first week that the trainee is in the 
Army. The AFQT score {in percentiles) represents a combination of four tests given at 
the time of the pre-ip.dut'tion physical examination. Table 1 lists the entry characteristic 
measures obtained for oach of the trainees. 

The Army Ai)titu(l(^ Area scores are combinations of ACB scores that have been 
previously valicIat(Hi for their ''relationship" with success in certain occupational areas 
within tiic Army. Aptitude^ Area scores are used as prerequisites for a variety of training 
programs For exainpl(\ tiie Clerical Aptitude Area (CL.) score is the average of the scores 
obUiincHi in ihc Vcrbnl (VK) and the Army CUTical Speed (ACS) tests. A CL score of 90 
is the pr(M*(Hjuisito for entry into thi* 7r)P20 course at QMS. Thus, the distribution of 
scores in tiiat Aptitude Area, and on the components of that index (VE and ACS) is 
truncated diK^ to tiic lowvv limit of 90 reciuir(»d for course entry. 

SELECTION OF CRITERIA 

VVluM^ tins project began in FY 1971, no segment of the 76P20 course had yet been 
individvialized. A plan was devised to use the available predictor variables (AFQT and 
ACB test seor'\s} to predict time-to-completion on those sections of the conventional 
course^ that wire -'seir-paeed/' in the 76P20 course, the graded practical exercise (PEE) at 
the (mk! of eaeii instructional block was worked on hy the trainees at their own pace. In 
the traditional classroom, it an indivkiual completed the PEE rapidly, he would assist one 
of the slow(^r tnembers of the class. In order to assess the predictive ability of each 
fai'tor. this praetii e was halted and each student worked by himself. The instructors were 
j>rovkled witii forms to record tiie start and completion time for each PEE. As a reward 
fv)r completing the PEE as rapklly as possible, students were allowed to leave the 
cUissroom ond have the rest of the period as free time. 

In addition to PEE completion time data, the pcvformcnce sc^ores obtained on the 
PEEs were collect(^d. Only those time scores for PEEs that wwe '^passc^d" (ke., 70/r 



Table 1 



Trainee Entry Characteristic Measures 

ACB (Army Classification Battery) 



Symbol ^ Test 

VE Verbal 

AR Arithmetic Reasoning 

PA Pattern Analysis 

CI Classification Iriventory 

MA Mechanical Aptitude 

ACS Army Clerical Speed 

ARC Army Radio Code 

GIT General Irsformation 

SM Shop Mechanics 

Al Automotive Information 

ELI Electronic Informal, j>n 

Army Aptitude Areas 

Symbol Title FVufiuiWi 

!N Infantry-Comtiat AR 4- 2CI 

3 

AE Armor, Artillery. GiT ^ Al 

Engineers- Com bat 2 

EL Electronics MA ♦ 2EI 

3 " 

GM General Ma'fUenance PA 2SM 

3 

MM Motor Maintenance MA 2AI 

3 

CL Clerical VE + ACS 

2 

GT General Technical VE ♦ AR 

2 

RC Radio Code VE + ARC 



AFQT (Armed Forces Qualification Tost) 



Percentile Score - Combination of Verbal, Arithmetic, Spatial Relations, and Knowledge of 
Tools (Mechanical). 



correct) were used as criteria in this investigatiop. This practice was adopted because the 
completion of a self-paced .course implies reaching a minimum criterion level of perform- 
ance. "In the present study this requirement could best be simulated Jby using PEE 
completion time figures for only those students who had obtained a passing grade on 
the PEE. 

In Phase 1 a question arose as to which would be a more valid criterion— completion 
time for a single PEE or completion time for all PEEs contained in the course. During 
Phase I, the 76P20 course was composed of three large instructional blocks preferred to 
as Annexes A, B, and C). Only the PEEs at the end of Annexes A and B were used as 
criteria in Phase I of this study. At the outset of the study, single PE^C times in 
Annexes A and B were used as criteria. As more data were collected, total PEE time 
within an Annex (J^ or B) was used as the criterion variablo. 

COLLECTION OF WITHIN-COURSE DATA 

\. ♦ 
The time 6nd performance scores for each of the PEEs were recorded on forms by 
the instructor in the classroom. (Data collection was not tightly controlled bec^iuse fiiany 
different instructors recorded these scores.) The forms were then forwarded to the 
HumRRO staff member stationed at QMS. He organized and forwarded the data to 
Alexandria, where it was put on IBM input slieets for keypunching. A card file wa^ set 
up for e^ch student. In addition to the PEE performance and time scores, the ACB and 
AFQT scores were transrnitted and entered into the file. 

ANALYSIS OF DATA 

Correlation matrices were th^n computed. The goal was to identify some of the 
factors that were likely to be unrelated to the criteria. Four 76P20 classes were used in 
the initial group of students (Group I J. At first, correlations of ACB, AFQT, and 
Aptitude Area scores with single' PEEs in each Annex (A and B) were computed. 
However, as trainees became more plentiful, the total time and iierformance scores across? 
PEEs for Annex A and for Annex B were used as criteria. Seventy-seven trainees had 
taken all the Annex A PEEs, and 76 trainees'had all the Annex B PEos. The correlations 
of the predictor variables with these (criteria ar^e showil in Talkie 2. 

Many of the correlations were statisticciliy significant (p<.05). Of the Aptitude 
Areas, GT and GM correlated approximately .5 with time and +.5 v>rith performance. 
This level of association was also found with the AFQT score. Certain individu"ffl^^CB 
tests (AH and PA) were as hif^hly related to those criteria as the combined scores. ^' 

The reliability of these findings was assesst^d with another group (Group II), con-^ 
sisting of three classes. Correlations were calculate^! and the results can also be found in 
Table 2. As can be seen, the size of th.* correlations lietweert predictor variables and 
Annex A and Annex B PFE performance score s was sustained. However, the relationships 
with time scores decreased. This reduction, which occurred for most of the predictor 
variables, could have been a result of the unwntrolleo factors previously mentioned (i.e., 
different instructors in d'ffercnt class;f»s recording' the time scores). 

In Table 2, correlations that ar^ circled represent correlations (not significant at the 
.05 level) of pr^^lictors with criteria across Groups. Thus, the correlation of CbfSsification 
Inventory' (CI) scores with criteriiCi both time and performance) never reached statistical 
significance. It wa: assumed, then, that this variable could be eliminated from further 



Table 2 



Correlations of AFQT, ACB, and Aptitude Area Scores With 
Annex A and Annex B Time and Performance Scores 





Group 1 


Group 1 1 




Annex A (N - 77) 


Annex B (N - 76) 


Annex A -~ G1 ) 


Annex 13 = 73) 


Predictor 


Perf orman ce 


Time 


Per f ormance 


Time 


Per^orman ce 


Time 


Per f orrnance 


Time 


AFQT 


.53 


-.46 


.51 


- IQ 




.63 


-.22 


.49 


- 41 


ACB 
VF 

V u 

AR 

PA 

MA 

ARC 

SM 

Al 


.45 
.58 
.62 
.39 
.26 
.40 
.30 


-.38 
-.54 
- .50 
-.35 
-.34 
... .44 

-.23 


.34 
.42 
.48 
.42 
.15 
.40 
.34 


- .28 
-.45 
-.37 
-.41 
-.33 
-.36 
-.30 




.52 
.64 
.50 
.38 
.20 
.17 
.19 


- .29 
-.18 
-.35 
-.33 
-.22 
-.20 
-.34 


.37 
.45 
.45 
.31 
.12 
.28 
.32 


-.25 
-.36 
-.40 
-.20 
-.11 
-.11 
-.14 


ELI 


.41 


( ~ ^lJ 


.42 






.39 


( -.32) 


.42 


(-.23) 


GIT 


.32 


-.38 


.34 


- .33 




.45 


-.38 


.44 


-.15 


CI 




-.21 


.20 


-.10 




C 14 


.13 


.00 


-.02 ) 


ACS 


-.30 


.17 


.11 


-.44 




.36 


-.34 


.17 


-.17 


Aptitude 
Area 

CL 

GT 

IN 

AE 

EL 

GM 

MM 

RC 


.43 

.56 
.35 
.33 
.46 
.57 
.36 
.39 


49 

-.50 
.38 
.36 

-.29 
.53 

-.30 
.41 


.30 
.43 
.34 
.36 
.47 
.48 
.40 
.28 


-.45 
-.42 
-.27 
-.35 

- 36 

- .43 
.36 
38 




,55 
,66 
,36 
.35 
,44 
,29 
.33 
.37 


-.34 
.26 
.06 
-.40 
-.37 
-.32 
-.37 
-.32 


.35 
.48 
.14 
-.43 
■43 
.37 
.41 
.25 


-.23 
-.36 
-.15 
-.16 
-.25 
-.25 
-.22 
-'.20 



consideration. Simihirly, KLI showed low correlatio!\s with time scores in both groups. 
However, it did correlate sij^nificantly with performance scores, and was retained along 
with the other .ACBs and AFQT scores. 

SUMMARY OF FINDINGS AND CONCLUSIONS 

The following conclusions were drawn from the correlation analyses in Phase I: 

(1) The individual ACB test scores correlated as well with criteria as did the 

combined Aptitude Area scores. 
(4?,) The Classification Inventory (Ci) score was consistently unrelated to any of 

the criterion varial)les (Annexes A and B PEE time and performance 

scores). 

(3) The AFQT usually provid(»d as high a correlation with criteria as any of the 
singl(» ACB tests. 
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decisions made on the basis of these findings were as follows: 

(1) In future analyses only the individual ACB scores would be employed. 
Aptitude Area scores would no longer be used. 

(2) The Classification Inventory score would not be included in any further 
analyses. 

(3) The AFQT and 10 other ACB test scores would continue to be obtained in 
the next Phase of this effort. 



chapter 3 



PHASE II - PREDICTIVE TEST BATTERY 1 

r 

DEVELOPMENT 



While the Phase I analyses were hoing performed (in FY 1972), the 76P20 course 
content was carefully reviewed, in order to develop the Supply-related test battery. 
Although there was no attempt to incorporate content directly into all of the test items, 
the tests did reflect the skills and knowledges required in the general area of Supply. 

Two major types of tests were developed. First, there were **aptitude-like*' tests- 
tests of skills and knowledges (e.g., arithmetic, vocabulary) considered necessiiry to 
^'enable** the student to acquii'e the MOS-related ix^haviors. These tests included items 
that were course content-related to some extent, but not exclusively. 

The second type of test is best described as a ''mini-lesson and test'' situation. Five 
of the test instruments in this gi'oup were developed by Sticht (19) as ^'reading tests*' for 
another Supply MOS(76Y). The other '*mini-lesson/test'* instruments reflected Supply 
tasks that were taught in the 76P20 course. 

The composition of Predictive Test Battery 1 is shown in Table 3. A discussion of 
each test in this battery follov/s. 

Table 3 



Predictive Test Battery 1 -Supply Area Related 







Nijmhxir of 1 tenis/ ^ 


Time Limit 


Tust Gfoui:) 


Tost 


Entries 


(Minutes) 


Arithmetic Skills and 


Number Recognition (NR) 


24 


2 


Knowledges 


Symbol Recognition and 








Operation (SR&O) 


44 


7 




Fractions and Decimals (FR) 


23 


5 1/2 




Word Problems (WP) 


21 


10 



Clerical Skills and Number Comparison (NC) 33 6 1/2 

Knowledges Filing and Alphabetizing (FA) 20 3 

Proofreading (PROOF) 25 ' 1 

Following Directions (F D) 25 7 
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Verbal Skills and Vocabulary {VOO 39 5 
Knowledges 

Sticht Reading Tests^ 

Reading Test A (RT-A) 20 5 

Reading Test B (RT-B) 5 2 

>^ Reading Test C (RT-C) ' 8 4 

XReadmg Test D (RT-D) 17 8 

Reading Test E (RT-E) 14 3 1/2 

'^See []% which (iescnbe^hose tests. 



ENABLING SKILLS AND KNOWLEDGES TESTS 



One test ^oup consisted of four tests that measured certain basic arithmetic skills 
and knowledges: 

(1) Numb er Recognition (NR) . This t^st assessed basic arithmetic and reading 
skills. It consisted of presenting a number written out in letter form (e.g., EIGHT), and 
then providing, in a multiple choice format, five alternative num.erical numbers. The 
student had to decide which of the alternatives was correct. At first glance, the required 
behavior may appear rather trivial. However, the test included items that had as many as 
six digits, and recognition was not always easy. 

(2) Symbol Recognition and Operations (SR&Q) . This test required the 
student to perform all the simple arithmetic operations (addition, subtraction, multiplica- 
tion, and division). In order to do this, he had to understand the symbols for these 
operations. For .\ample, 4YZS had to be recognized by the student as a division 
operation. Then, he had to select the correct answer from the alternatives listed. Thus, 
there were two skills necessary for successful performance (recognition of the arithmetic 
operation symbol, and accurate performance of the operation). 

(3) Fractions and Decimals (FR). This test measured the student's ability to 
distinguish between the values of fractions and/or decimals. In this test the student was 
presented with a set of five alternatives composed of all fractions, all decimals, or some 
of each. The student had to select the **smallest*' value of the alternatives'^presented. 

(1) Word Problems (VVP) . This test assessed the student's ability to perform 
simple arithmetic operations in order to solve problems that were presented in narrative 
form. Thus, reading comprehension skills were also involved. Most of these problems were 
related to the Supply ai'ea, and all were Army-oriented. After he read each problem, the 
student had to select the correct answer from the alternatives presented. 

Two "'aptitude-like'' tests were developed, to assess certain basic clerical skills and 
knowledges: 

(1) Number Compai-ison (NC) . This test assessed the ability of students to 
accurately compare a given number, several digits long, with other simihir numbers of the 
same length, in order to find a ''match.'* This skili is relevant to many of the ^'editing" 
tasks learned in the 76P20 course, such as checking 11-digit Federal vStock Numbers 
(FSNs) which are frequently used. 

(2) Filing/Alphab(> tizing ( FA). This test asse.s.sed knowledges regarding the 
correct letter sequence to employ when listing items in alphal)etical order. The student 
was provided with five similarly spelled items of equipment, or topics of military interest. 
Then he was asked to select' the item that would come first if he were to urder these 
words alphabetically. This knowledge is of piirticuiar importance in the vSupply area, 
where many indices, catalogs, and files are organized al|)habetically . 

One enabling skills and knowledges test was concerned with verl:)al al)ility: 

Vocabulary (VOC). This test was formatted as a typical multiple-choice 
vocabulary test. The only difference was that the items were Su|)ply area-related words 
(e.g., requisition). For each word presented to the student, he had to select the word or 
phrase that came closest in nieaning. Thus, the test ass(^ssed the k'vi'l of knowk^dgc^ in the 
MOS vocabuku-y that the student had when he entered the course. It was assumed that 
this should have had a direct relationship to th(* learning rate in tlie course. 

CONTENT RELATED TESTS (MINI-LESSON/TESTS) 

This type of instrument is, in reality, a miniature learning siluaLion (lo). in general, 
a situation to be eiicountered later in the course was simulateti in a miniaturized form 

O 
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and presented to the student. \ small amount of instruction in each task was given to 
the student, who then had to take a ^^test" on what he had just learned to do. 

Two 76P20 content-related tests-; which emphasized certain ^^clericar* behaviors, 
were developed: 

(1) Proofreading (PROOF) . This mini-lesson/test instructed students and then 
measured their ability to edit parts of Army Supply documents (forms). This task is an 
important one in the 765-20 course. A form with certain entries was presented to the 
student. Below this first form, another containing similai* information was shown. The 
student*s task was to determine whether the information was recorded properly on the 
second form, by comparing the entries with those on the first form. In some cases the 
student needed only to state whether each entry was correct or incorrect, in other cases 
he was expected to be more explicit. A multiple-choice format was used. 

(2) Follow i ng Directions (FD) . This instrument assessed a student's ability to 
follow step-by-step instructions in recording information. A Supply form was presented 
to the student. His job was to select the correct information and enter it in the correct 
space on the form, according to the written instructions provided. Thus, reading compre- 
hension skills were also critical to this task. 



STICHT READING TESTS (A-E) (RT-A THROUGH RT>E) 

Although grouped together in Table 3 as verbal-related instruments, the Sticht 
Reading Tests required the use of skills other than reading (e.g., **seai'ching'* behavior, 
simple arithmetic, and alphabetizing). These tests are grouped in the •''mini-lesson/test" 
category becaus(^ a ^*leiirning" experience was included in each situation.' The same 
general procedure was followed for each test, but there were viiriations in the content, 
the number of items, and the time limits required for completion. In each situation, a 
portion or extract of an Army document (e.g., Army Regulations, catalogs) was displayed 
on the left side of a folder. The specific task to be performed was then described, and an 
example was firovided. Answers were to l)e written on the answer sheets attached to the 
right side of the folder. Five such instruments Were used, the content of all were in the 
Supply area, but none was dirertly n-laLed to the 76P20 course. 



TEST BATTERY TRY-OUT 

Once the instruments described above w(^re constructed and organized into the three 
groups shown in Table 3, pilot testing was conducted at HumRRO Division No. 
1 (Alexandria) and at QMS. The main purpose of these administrations was to set time 
limits for each of the t^sts in the battery. The shortest time it took anyone in the pilot 
group to complete the test was used as the time limit for that inst/uir.ent. Since aJ] .those 
in the pilot group were of above average ability, it was assumed tha* at least 907r. of the 
students in the population would not complete the test in that time. 

These pilot try-outs also permitted a "shake-down'' of the administrative require- 
ments of the battery. Necessary changes were made to the -ambiguous items on the tests 
and to any unck^ar instructions. The test battery was then printed and prepai-ations were 
made for its administration. 
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ADMINISTRATION OF TESTING PROGRAM 

The first predictive test battery was administered in 1971. The students who 
participated in the 76P20 course at that time were approximately 60% draftees and 40% 
Regular Army and National Guard members.' 

Arrangements were made with QMS personnel to set aside a classroom for the 
approximately two hours required for testing. Students, reported in at Fort Lee on a 
Friday and were tested on the following Wednesday morning. 

As they arrived in the classroom, students were given^all the necessary materials. The 
HumRRO staff mem.ber then read a general set of instructions. Some of the points made 
in these directions were the following: 

(1) Students' performance on the tests would not affect their progress in the 
course or in the Army. 

(2) They should work as rapidly and as accurately as possible. 

(3) Each test had a time limit and no one was expected to complete all of the 
items on each test. 

(4) They should ''guess" in a multiple-choice situation only if one or more of 
the choices could be eliminated, as there would be a penalty for incorrect 
responses (this statement was inserted to minimize guessing, even thougli 
the total number correct was used as the index for predictive purposes). 

The first section of the test battery (Arithmetic Skills) was then presented. Instruc- 
tions were read and the example item worked on by everyone. This procedure was 
followed for each test in the battery. After the Arithmetic Skills section was completed, 
the students were given a ten-minute break. Then the Clerical Skills section was 
presented, followed by another ten-minute [break. Finally, the third section (Verbal Skills) 
was administered. The Sticht Reading Tests were presented with a prefatory statement 
that prepared the students for their probable unfamiliarity with this type of test. The* 
time limit of each test was rigidly enforced.V^ 

After completing the testing: program, students entered the 76P20 course. 

\. 

SELECTION OF CRITERIA 




At the time this study was being performed, no instructional segment of the course 
had been completely self-paced. Therefore, the same criteria used in Phase I (i.e., time to 
complete Annex A and B PEEs) were employed. However, Aprr^x A or Annex B PEE 
total time crould not be used as a criterion because only 15%/of the students in Phase II 
participated in all PEEs. Th^ft^re, individual PEE scores w^re employed in the analyses 
in this Phase. \^ i 

• C'' .^'^^-^ / 
COLLECTION OF WITH IN-COURSE DATA / 

The same procedures used in Phase I were again followed. Arrangements were made 
with the instructors to collect the criterion data (Annex A and B PEE times and scores). 
The need to be accurate in making these recordings was emphasized repeatedly. Also, as 
l^efore, background data were obtained on all students. 

r 

'These proportions stayed osscntially the same through Phasf,' 111. 
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ANALYSIS OF DATA 



As in Phase I, correlation matrices wore computed. However, total PEE time and 
performance were included amon^? the variables. Although five 76P20 classes v/ere used in 
tnis Phase (N = 151), only 20 students had scores for all Annex A PEE components. 
Thus, lh(* analyses concentrated on PEE components, one at a time, ' 

DEVELOPMENT OF PREDICTIVE EQUATIONS 

Stepwise multiple linear regression analyses (20, 21, 22) were performed on the data 
to derive the predictive ecjuations. This is a technique that selects variables for a linear 
regression equation, one at a time. It begins with a simple correlation matrix, and enters 
into regression the predictor variables most highly correlated with the criterion variable, 
A linear equation is {)roduced, which includes a regression coefficient that is to be 
mulli[)lied to the value of the predictor variable, plus a constant. 

'I'his analysis also provided a test (Partial F—Analysis of Variance) of the '^sig- 
nificance'' of the variables in the equation. This was, \n fact, a test of the significance of 
the regression eot^fficients. (Was the **true** regi'ession coefficient significantly different 
from zero?) If, at a given confidenc:e level, the null hypothesis could not be rejected, 
then the regn^ssion coefficient was assumed to be zero. A zero coefficient, multiplied to a 
given ()r(»di( tor value, would result in zero or no contribution of that variable to the 
equation. The variable would thus be prevented from entering the equation. 

Another way of looking at this procedure is in terms of the correlation coefficient. 
In the ew^e of the first variable to enter thf^ A^quation, it is a simple correlation 
coefficient. With more than one predictor, a multiple R is calculated. The coefficient of 
determination {R~ ) indicates the portion of the variance in the criterion variable 
distril)utic)n accc)unted for by a given predictor variable (shared variance common to both 
variables). After the stepwise multiple regression analysis selects the predictor variable 
that is most highly com^lated \yith the criterion variable, it then selects that variable 
whic'h, when combined with the first, is the most useful— that is, the one that adds the 
most to the multij^le correlation coefficient, and that yields the best two-predictor 
equation from among the possil)le equations containing ' the first variable selected. A 
partial corr(*lation coeffic ient is calculated for each of the predictor variables. The second 
variable to enter the equation is Ihv one with the highest /?ar^/a/ correlation with the 
criterion variable. However, it will enter the equation only if the partial F test for that 
variable is significant at the probability level selected. 

The technique then selects (by the same criteria) th(^ v^U'iable that, combined with 
the first two variables, [)roduces the best three-[)redictor equation. Subsequent valuables 
are selected in a similai* manner. Variables can also be removed if they are found to be 
no longer useful. This is done by evaluating i*ach variable's, contribution as if it had 
entered last. Any variable that provides a non-significant contribution is removed from 
the equation. This process is continued until no more variables will be admitted to the 
equation and no more will be rejected. The process then terminates aild presents the last 
''best** regression equation, given the imposed constraints' (i.e., when adding the most 
useful remaining variable produces no statistically significant increase in the multiple 
correlation according to a pre-selec^ted confidence level). An example of the output of 
such a computer analysis is shown in Appendix A. 

After several attempts using the least restrictive level it becan-je clear that an 
optimum approach to employ in selecting the appropriate confidence level was as 
follows: A confidence level of p = .80 was used by the computer program to . allow 
variables to enter or leave an equation. In ^ome cases the computer program generated ai-i 
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equation with 10-12 predictors in it. Following the selection of the **best'' equation by 
the computer program, a visual inspection and screening was made at the ..95 level of 
confidence, for the purpose of obtaining an equation with a smaller, more efficient set 
of variables. 

In this Phase, equations were generated to predict each PEE time and performance 
score. As stated above, no total time criterion was obtained. Therefore, different sets of 
trainees were involved in each separate regression equation. An examination was made of 
the order of entry of each of the variables into the predictive function, to see whether 
any commonalities existed from one PEE criterion to another. In this way, an indirect 
attempt was made at determining the more probable "important*' predictors. 

• On the basis of these observations, plus careful scrutiny of the correlation matrices, 
several variables stood out as highly probable **important'' predictors. These were as 
follows: 

For Predicting Time Scores For Predicting Per formance Scores 

Symbol Recognition and Operations Symbol Recognition and Operations 
Reading Test A' AFQT 

Reading Te'st Arithmetic Reasoning (ACB) 

Follov/ing Directions Verbal (ACB) 

Vocabulary Proofreading 

Arithmetic Reasoning (AGB) 
As this list shows, there are some tests that were appropriate for performance and 
time (SR&O, AR), whereas others were more related to one criterion than another. In 
•predicting time scores, the tests in Predictive Test Battery 1 did much better than the 
ACBs. However, the ACBs did as well on performance. This should not be surprising as 
the HurnRRO-developed tests were designed to predict learning time. 



SUMMARY OF FINDINGS AND CONCLUSIONS 

The following conclusions were drawn from the analyses in Phase II: 

(1) The HumRRO-deve loped test scores correlated more/ strongly with time in 
Annex A and B PEEs, than did the ACB scores. 

(2) The Fiaction.s and Decimals test did not show any predictive utility and 
was dropped from further consideration. 

(3) Stepwise multiple linear regression analyses were employed in the 
development of predictive functions and the determination of potentially 
useful predictors. 



'J 

"The sum of scores on a!! five reading tests was the most powerful correlate with criterifin (time). 
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Chapter 4 ^ 



PHASE III . PREDICTIVE TEST BATTERY 2 

DEVELOPMENT 

Although Predictive Test Battery 1 was Supply-related (in terms of the broad^ 
content area), an attempt vyas made in this Phase to develop instruments that were more 
directly related to the course undergoing individualization (MOS 76P20). This activity 
involved a very detailed examination of the instructional materials so as to identify 
specific objectives that could be sampled in Test Battery 2. To assess capabilities relative 
to these objectives, additional content-related tests were developed. The v£irious test 
components of Predictive Test Battery 2 are described below. 

CONTENT RELATED TESTS (MINI LESSON/TESTS) 

At the time of Phase 111 (late FY 1972) the Stock Control and Accounting 
Specialist Course (76P20) was divided into iTttee instructional annexes (see Table 4, 
which shows current structure of the 76P20 course).^ Annex A had six subsections; Annex 
B had six subsections; Annex C (since changed tjxD)' had four subsections. The results 
of Phase II indicated thai the Sticht reading tests were potentially useful predictors. It 
would tx^ logical, then, to replace the original tests only with other similar instruments 
that contained tasks that wc-e clearly representative of the tasks that students would 
learn to perform in the 7()P20 course. As a number cf modifications were l>eing made at 
tha time to Annex C (ASDA Accounting PnurduresK no predictive instruments were 
developed that related to that portion of thi^iontent. 

The basis for developing the ten mini-lesson /tests (labeled "Reading Tests I-IC) was 
the instructional material in Annext^s A and IV The tourse objectives were determined 
from the tests, on whic^h the studejils wen* «ira<i(*d during, and at the end, of the course. 
In other words, the approach was to find out what the studeiits had to know in order to 
pass the course, an^ from that [)oint (lev(»lop the mini iesson/test situations. 

As the first segment of the course to be individualized was in Annex B, a large 
mini lesson /test was constructed (FKB) to sample the l)ehaviors associated with that 
Annex. This test spe iTically covered tasks in the individualized pojtion (B-'A) of the 
Annex. For this reason, the VEB .'xpected to a good prei(ictor of student 

performance in B-.'V In format, it was simihu* to the other 10 mmi-lesson/tests in this 
section of the l)attery. 

ADDITIONAL ENABLING SKILLS AND KNOWLEDGES TESTS 

One additional clerical skills and knowledges test was constructed for Predictive Test 
Battery 2: 

Abbreviatio. (ABBRKVl. This instrument measured the student's ability to 
retognize c^ommon abbreviations used in documents or forms for the 7(iP2() course. 

' Witli tlu' inclusion of Mr(h;ini/r(i Slot-k Control srriion as Annox C. ;\SI),\ Accounlin^ 
Proi (lures'^;! s relalu'led as Annex 



Table 4 

76P20 Coarse Instructional Content by Subject^ 



— — j 

Annex | 

1 


Major Topic j 

! _ 1 




Sub-Topic 


A 


Establishing a Stock 


• 


Establishment of a Technical Library 




Record Account 




Verification of Equipment Density Lists 






• 


Review of Prescribed Load Lists 








Compilation of ^n Authorized Stockage List 






• 


Preparing Stock Accounting Records 






• 


Editing Request Documents 


B 


Management of Stock 


• 


Processing N on Stocked Item Request Documents 




Records 


• 


Processing Low. Priority ASL Requests 






• 


Prqcessing High Priority ASL Requests 






• 


Processing Inventory Documents 






• 


Processing Excess Property 








Accouriiirjg tor oiass using iuud ia nepair rdiib oybitiTi 




Mechanized Stock 


• 


Purification of Manual Records 




Control at DSC 


• 


Conversion (L^ata Preparation) 






• 


Document Processing 






e 


Systems Programs 






• 


Equipment Familiarization 


D 


ASDA Accounting 


• 


Processing Requisitions and Requisition Modifiers 




Procedures 




Processing of Follow-up and Cancelliation Documents 






• 


Processing Replenishment Documents 






• 


Processing Material Adjustment Documents 



^Extracted from U.S. Arnriy Quartermaster School, Program of Instruction for 551-76P20 • Stock Control and 
Accounting Spcciali&t, MOS: 76P20, February l972. 

^This annex was formerly a separate course and only recently was incorporated into the 76P20 Program of 
'nstrurfion. For this reason, no course-related x^redictive measures had been developed for this ar»nex as of the dale 
of this report. 



"AFFECTIVE" INSTPiUMENTS 

In addition \p the skills and knowledges tests, a set of ^'affective" factors considered 
important in the jTrediction of classroom behavior were assessed by Test Battery 2. These 
^'motivational" characteristics have been described in the literature as contributing to 
classroom performance. However, they are not easily measured. An attempt was made in " 
this study to devise sWeral instruments that would get at this '^affective" dimension, and 
thus improve the preaictive equations that incorporated these factors. These instruments 
are described: / 

(1) Spiire Time Interest. This consisted of a list of 15 spare time activities. The 
activities ranged from passive (^nes such as watching TV or listening to the radio, to more 
active ones such as playing sports or fixing things. The student was asked to rate^ on a 
seven-point scale, his interest in participating in the various activities. (The range of this 
scale was from '"cotnplete disinterest" to ''complete interest" in performing a particular 
activity.) In addition, the trainee was asked to rank the 15 ^'activities from ''Most 
Preferrec!" to "Least Preferred". Thi^ instrument served as an indirect assessment of 
.nstructional media preference (as the course was to have a multi-media format). 
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(2) Method of Instruction. This instrument, consisting of a list of 10 instruc- 
tional methods, was similar to the one just described in that it required the student to 
make both ratings and rankings. With this instrument, however, the student was asked to 
directly rate the various means by which instruction might be presented to him, (e.g., 
programmed text, lecture). Again, a seven-point scale was used, ranging from "dislike 
completely'' to ''like completely." He was then asked to rank the ten instructional 
methods in order of preference. 

(3) Definition Preference . This instrument might best be described as an 
indirect measure of the preference of students for the subject matter areas covered in the 
Stock Control and Accounting Specialist course. The student was shown a word that has 
four valid definitions, one of which was Supply-related. He was then asked to order the 
four alternatives in terms of his preference (Numl>er 1 was used for the most preferred 
and Number 4, the least preferred). The critical score was the number given to the 
Supply-related definition. Scores could then be related to actual course performance. This 
instrument contained a total of 20 words, each with their alternative definitions. 

(4) Su pply Activity . This test consisted of 15 paragraph descriptions of the 
important tasks that would be covered in the course. The student was instructed to read 
each pjiragraph, and rate it according to his interest in performing that activity. He used a 
seven-point scale ranging from ''complete disinterest" to "complet-e interest." When he 
finished this part of the test, he was asked to rate hoiv well he thought he would 
perform during the course on each of the 15 tasks. A five-point scale ranging from "fail" 
to "excellent" was used for this level of aspiration measure. 

The 76P20 course was to be developed as a "multi-media" program. That is, both 
programmed text and AV (slide/tape) versions of the instruction were to be prepared. In 
order to b. tter predict learning rate in the slide/tape mode, an attempt was made to 
develop "AV" -related tests. This was accomphshed by converting two of the reading 
tests, Reading Test-4 and Reading Test-8, to Listening Test-A and Listening Test-B, 
respectively. The student was provided with the reference information on the left hand 
side of the folder as in the reading tests. Now, however, the questions were presented by 
means of a tape recorder. Each question was repeated twice, and then the tape was 
stopped. The student answered the question on his answer sheet and raised his hand as 
soon as he had finished the item. When two-thirds of the class had raised their hands, the 
tape was started for the next item of the test. No time limit v/as imposed on either test. 
However; Listening Test-A required approximately seuen minutes and Listening Test-B 
about six minutes to administer. 

The tests in Predictive Test Battery 2 are listed in Table 5, along with their time 
limits and the number of items in each instrument. Several minor changes were made to 
the tests that remained from Test Battery 1. Mainly, these consisted of deleting, adding, 
or substituting items to make the tests more course-related. Also, the directions to the 
student were revised to make them clearer. A comparison of Table 3 with Table 5 shows 
the changes in the number of items or in the time limits of these tests. Pilot groups were 
run again to determine the new time limits, using the same procedure followed for 
Predictive Test Battery 1. 

ADMINISTRATION OF TESTING PROGRAM , 

The second test battery was administered late in FY 1972. By this time the QMS 
had instituted its veision of "zero week," which meant that details and other duties were 
to be performed during the first week that the trainee was on post. Following this first 
week, he would be in an academic status, not to be removed for extraneous jobs. 
Fiowever, execution of this principle did not always meet original expectations. Arranging 
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Predictive Test Battery 2-Supply Course (76P20) Related 
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48 
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Vocabulary 


39 


4 


Knowledge 


Reading Test-1 


20 


7 1/2 




Reading Test-2 


23 


4 1/2 




Reading Test-3 


6 


2 




Reading Test-4 {LT A)'^ 


2) 






Reading Test-5 
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O 1 '0 
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1 7 


8 




Reading Test-7 


Q 














Reading Test-9 


9 


3 'i/2 




Reading Test-10 


12 


1 




Practical Exercises-B 


29 


12 1/2 


Affective 


Spare Tinne-lnterests 








(Rating and Ranking) 


14 


None-(IO)^^ 




Method of Instruction 








(Rating and Ranking) 


10 


NonG(5)^ 




Definition Preference 


20 


None-(15)^ 




Supply Activity (Rating. 








Ranking, and Level of 








Aspiration) 


15 


None-dO)^^ 



^Ong]na\]y constructed as reading tests, RT-4 and RT-8 were put on tape and used as L>sten/f)g Tests A arjd B, 
respectively. 

^No lirne limits imposed on these tests. Average lime required to complete each test is in parenthesis. 



for test administTation wjis difficult during '*zero week*' as the trainees were under the 
auspices of the Brigade Commander, not under QMS academic department personnel. 
Also, a5 can be seen in Table 5, Test Battery 2 was considerably longer than Test Battel^ 
j. Approximately four hours were needed for testing, and this time was difficult to 
obtain. 

The trainees were tested in the afternoon. 'I\n-minute breaks were given at the end 
of each of the first three sections of the test battery. The general instructions that were 
read to the students were similar to those described under Test Battery 1. The 
Q **Affective'' tests were all self-paced and given at the end of the session. That is, aft(T 
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completing,' the verbal skills tests, individuals were told to complete the ^'Affective'' 
instruments at their own pace, after which they could leave. 

SELECTION OF CRITERIA 

The first segment of the 76P20 course was individualized during this Phase of the 
study. This course segment is referred to as B-3 (the third section of Annex B). The 
question arose as to what would be the appropriate time scores to use as criteria for 
predictions. Although toiaal time in the entire course is the ultimate criterion, there was 
no evidence that total time in the B-'8 segment was- reflective of the course as a wiiole. 
Therefore, a decision was made to initially use multiple criteria, and then empirically 
determine which functions had the greatest utility. 

The B-3 course segment was composed of five instructional modules, an ungraded 
comprehensive practical exercise (CPE), and a graded practical exercise (PEE). The 
instructional modules were presented by means of a programmed text or slide/tape 
(Kodak Cai-ousei 650 projector coupled with a Norelco cassette tape recorder).^ Each 
module ^ilso had a practical exercise segment in which the student could check his own 
state of proficiency. He could then go on to the next module or, if necessary, repeat a 
module. QMS instructional monitors would assist the students if any mechanical problems 
occurred with a given tape or slide unit. 

The STOCK staff and QMS personnel responsible for developing and administering 
the self-paced course prepared recording forms to be used in the administration of this 
course segment. The start and slop times for each module, CPE, and PEE were recorded, 
along with the time taken for breaks by the student. Various combinations of .these times 
served af. the criteria in attempts at prediction. However, the most useful criteria were: 

• The sum of the times in all five instructional modules = Instructional Time; 

• Instructional Time plus CPE and PEE time - Total Time. 

COLLECTION OF WITHIIM-COURSE DATA 

The within-course PEE times and scores collected during Phase 11 continued to be 
collected in this phase. The ultimate ''course-related** predictor is, by definition, an 
earlier section of the course. It was expected that the PEE segments in Annex A and the 
first one (B-2) in Annex B (prior to the individualized course segment) would increase 
the predictive accuracy of the equations that were generated using these factors. Thus, 
equations were derived both with and without these previous course segment scores. 

ANALYSIS OF DATA 

As in the previous Pha.ses of the study, correlationaj and stepwise multiple linear 
regression analyses were performed upon the data. However, valid criterion variables (time 
scores from an ac-tuai self-paced course segment) were employed in this Phase for the first 
time. Predictive functions were developed, using the regression analysis technique 
descrilx^d previously, and then evaluated. 

Idonhfication of produces is for rcsei\rch dncumeniii\.\o\ purposes only ; this Ust'ini^ does not 
constitute an official ondorsemont by either HumRRO or tho Department of tlu- Amiy. 
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EVALUATION OF PREDICTOR VARIABLES AND EQUATIONS 



Initially, equations that usoci all of ihc data for all of tlio stud(»nts were ^^eiioratod. 
One surprisui^ finding was a very iow multiple torrelation eoeffieienL MR (N ^- 

i.'Ki). in Phase II, most of the nnilli|)le eorrelation eO(Tfieienls obtained witii Predictive 
Test Battery 1 upon PEE scores were in the to .7 range. How eould a more 
eourse-reiated ^oup of predictors produce a sDuiller association with criteria? 

In the resulting investigation, the data were grouped along several dimensions. By 
grouping the data into the mode of imtriictioti received (AV or PI-t4?xt), a startlinj.^ 
increase in association resulted. This occurred across classes, thereby increasing confidence 
in the finding, histead of a multiple (*orrelation of .31, multiple correlations of .67 and 
.75 resulted when using the data for AV (N ^ 17). and Pl-text (N ^ 80) groups, 
separately. 

Thus, in order for uceurate predictions to be made in an ifidividualized classroom^ 
the instructional situation (mode/media) is critical for determining learning rate, when 
there are significant differences in training time depending upon mode of instruction. 
This finding is supported in the literature cited earlier (T5, 16)- The results indicated that 
the AV-instructed students completed the course segment more rapidly (Xy^y = 10 
hours) than those receiving Pl-text (Xpj ^ 12 hours). As tliese two populations differed 
significantly, a different predictive function was needed for each group. It should be 
noted that, although the PTtext group required 12 hours for this course segment, this 
was still considerably less time than that allocated in the conventional course (IG hours). 

Table 6 lists the '*best*' predictive equations developed during this Phase of the 
study. Two sets of equations were developed—one set for AV-instructed students, and a 
second set for those who received their instruction in the P{-text. Most general aptitude 
measures (ACBs) did not enter significantly into these functions. However, the AFQT 
score was a significant component of each of the equatioiis. As can be seen in Table G, 
the predictors that contributed most to the equations were some of the previous PEE 
times in the course. For example, the A-3, A-5, and B-2 times were the most important 
predictors in Equations 3 and 4. The fact that previous within-couhe component times 
were highly significant supports the hypothesis of course-relatedness providing the *'best*' 
prediction of completion time. 

The **best" functions that did not include previous course component times are 
presented as Equations 1 and 2. In short courses (under eight weeks), it is nec(\ssary to 
make completion time predictions prior to th»' start of the course. The only eciuations 
that can Ix? used in that situation are those that contain extra-course variables. Thus, only 
the factors in Predictive Test Battery 2 and the student's background data were used to 
develop Equations 1 and 2. However, Table G indicates that, if a prediction can 
legitimately be made during course participation, earlier course component times should 
be employed in the derivation of the predictive function. 

Another finding of interest was that Total Time was not as accurately predict(»d as 
the Instructional Time of the segment. For example, although tin- mu]tii:>le c-orrelation for 
the Pl-text group was .85 using Instructional Time as the criterion, it only reach(^d .7S 
with Total Time as the criterion. 'I'his depressive effect (see findings in Tabic* 7) may be 
due to the test-taking portion of the Total Time criterion. To chi'ck on this, the *'best'' 
predictors uncovered by the regression analyses on instructional Time and Total Time 
were used to predict ''test-taking time." The resulting MRs vari(»d between .3 and .5. ^ 
When a different set of predictors were employed in the regression analysis on test-taking 
time (PEE time), the A/Ks were up to .65. 
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Table 6 



**Best" Predictive Equations Selected in Phase 111 





Predictions To Be Made Prior To fEntry In Course 






For Pi-Text Group 






Y = - .0646A - .2909B - 1. 1235C - .1606D + 37.678 






Key: Y-Training Time (days) Standard Error = 


3.778 (days) 




A A COT 






B-RTl MR = J± 






C-RT3 N = 81 






D-PEB 




(2) 


For AV Group 






Y - ' . 0774A - .205GB - . 0978C + 32.595 






Key: Y-Training Time (days) Standard Error = 


3.533 (days) 




A-AFQT 






lj-i>jl- r/in — .DO 






C-VOC N - 52 






PrpHir't ir\nc 1 ^ R p fsA ^jHp niirin*^ Triiriinn 




(3) 


For Pi-Text Group 






Y = - .0553A- 1.0781B + .1807C -f .07860 + 14.93 






Key: Y-Training Time (days) Standard Error = 


3.006 (days) 




A-AFQT 






B~RT3 MR^m 






C-A-5 (PEE) Time N - 67 






D-B-2 (PEE) Time 






For AV Group 






Y - - .0722A f . line + .1274C -f- 11.704 






Key: Y-Training Time (days) Standard Error = 


2.08 (d^ys) 




A-AFQT 






B-A-3 (PEE) Time MR = .87 






C-B-2 (PEE) Time M = 47 





A further investigation needs to be made to uncover the reasons behind this reduced 
capability to predict test-taking time. However, even with PEE time included, the 
resultant MR of .78 with Total Time compares quite favorably v^th the results of other 
studies involving training time predictions. These findings have shown that the content- 
related variables of Test Battery 2 are better predictors of **time-to-learn** than the 
general aptitude measures, thus confirming the results obtained in Phase li. 
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Table 7 



Multiple Correlations Obtained With Instructional Time and 
Total Time in Course as the Criteria for Predictions 





Criterion 




Instructional Time 


Total 


Time 


i 


PI -Texl Group 


AV Uioup 


Pi-Text Group 


AV Group 


i 

Prior to tntry Predictions (\>/ithout course 
component time scores) ' 


.744 
(N = 81) 


.650 
(N - 52) 


.671 
{N = 76) 


.634 
(N = 52) 


Within-Training Predictions (with course 
Component time scores) 


.852 
(N - 67) 


.871 
{N = 47) 


.780 
(N = 76) 


.824 
(N = 52) 



The tests within Predictive Test Battery 2 that contributed significantly to the 
functions developed and used in this Phase, are listed below: 
Number of recognition (NR) 
Symbol Recognition and Operations (SR & O) 

Number Comparison (NC) 
Proofreading (PROOF) 

Vocabulary (VOC) 
Reading Test 1 (RTl) 
Reading Test 2 (RT2) 
Reading Test 3 (RT3) 
Reading Test 6 (RT6) 
Reading Test 7 (RT7) 
Reading Test 10'j(RT10) 
Practical Exercis^B (PEB) 

Supply Activity - 'Rating and Level of Aspiration 

DEVELOPMENT OF PREDICTION DISPLAY TECHNIQUES 

PREDICT Computer Printouts 

One of the reasons for developing the predictive system described in this report was 
to provide training managers with information they could use for administering a 
self-paced training program. A computer program was written to display this prcdiciivo 
information in a useful format. The program (labeled PREDICT) displays each student^s 
estimated completion date (see Figure 1). 

This was accomplished by applying each predictive equation to the data base from 
which it was derived. As the criterion data consisted of the time (in minutes) within the 
B'3 course segment, a conversion factor was used to extrapolate these estimates into 
days. For example, there are 31,25 training days scheduled in the standard course for 
instruction in academic subjects. Instructtcnal Time in B-3 is 720 minutes. Total Time 
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(including CPE and PEE) in B-3 is 900 minutes. The conversion factors then were as 
follows: 

For Instructional Time - 



Mean Predicted Instructional Time (in minutes) ^ ^ 
720 31.25 

For Total Time - 

Mean Predicted Total Tirne (in minutes) ^ X 



900 31.25 

In this way, the student's predicted time (in minutes) in B-3, was converted to simulated 
predictions of days within the entire course. 

As can be seen in Figure 1, the PREDICT printout displayed the probability of 
obtaining a certain estimated completion time. This was based on the standard error of 
estimate (SE) statistic calculated in the multiple regression analysis. In Figure 1, the 
estimated completion time for Trainee B was 20 days. The probability that the estimated 
completion time was within the interval of 18-21 days was 50%; the probability that it 
was within 15-26 days was 99%. Information displayed in this manner could be used by 
training managers at QMS to estimate, with varying degrees of confidence, how much 
time each individual would spend in training. They couid also estimate the completion 
time distribution for efitire classes on the basis of this information. • 

For each person listed in Figure 1, both estimated arid actual completion times are 
shown. The predictive accuracy of the functions used to generate the "estimated com- 
pletion time" was assessed by applying each equation to the data base from which it was 
derived. Each student's estimated score was compared to his actual score and *the residual 
error was determined. For example, if the estimated completion time was 18 days for a 
given student, and his actual completion time was 20 days, then the residual error was 
two days. By calculating the number of **hits" and the size of this residual error, the 
accuracy of each predictive equation could be ascertained. 

Ideally, to validate an equation, it should be applied to a new sample. Although this 
approach would have been preferred, time restrictions and course development constraints 
did not permit the employment of such a controlled approach. Instead, a "split-half" 
simulation of such a validation design was used. That is,. the data from half the students 
were used to generate an equation. Then the equation was applied to these students using 
the PREDICT program. The MR and residual errors were computed. The equation was 
then applied to the other half of the population. The resulting associations (MRs) and 
residual errors were compared with those from the first group, and these comparisons 
indicated the reliability of the particular equation and its compo^ient variables. Each of 
the equations were '^validated" in this manner and the "best" equations were selected. 
These functions (shown in Table 6), had MRs that remained as high on the **new" ' 
individuals as they were on the data from which they ^eye derived. Also, their predictive 
accuracv (as determined by the residual error comparison)' ^id not diminish. 

Nomographs f^^^^^ 

Although the PREDICT display program is Ian acceptable technique for training 
managers to apply to self-paced courses, access to k computer is required to produce the 
PREDICT printouts. There is a need for training /managers at the QMS to be 'able to 
efficiently use the predictive equations, without the^necessity of a computer. 

Nomographs provide this capability. Nomographs are charts that express, in graphic 
Q form, the relationships that exist in a regression equation. The concept .of presenting such 
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relationships in nomographic form is not new and has been used widely in engineering 
(23, 24). However, its use in the behavioral sciences and education has been 
infrequent (25). The value of a nomographic representation of a predictive equation lies 
in its simplicity and the speed with which the information can be obtained by the user. 

Tv/o of the "best" regression equations selected in this phase were translated into 
nomographs (Figures 2 and 3). These nomographs—one for AV, the other for PI~text— 
permit the training manager at the school to determine the completion time for each 
student who has the scores required by the predictive equation. As each score is 
obtained, it can be entered in the appropriate scale and lines drawn in the appropriate 
sequence to the criterion line (completion time). All that is needed is a straight edge to 
draw the lines between the scales given on the chart. (The instructions for using each 
scale are presented in Figures 2 and 3). 

As each nomograph is unique, the sequence of connecting the scales by 'the various 
lines, as well as all other instructions, needs to be presented vrithin the chart itself. The 
nomographs presented in this report are simulations of the charts that will be developed 
with the total "individualized'' 76P20 course. As only a course segment (B-3) was used to 
develop the techniques described in this report, the nomographs depict only extrapola- 
tions from the data that were obtained. 

. SUMMARY OF FIIMDINGS AND CONCLUSIONS 

The critical findings in Phase III of the study were as follows: 

(1) Course content-related predictors were better than general aptitude 
measures for predicting completion time in a self-paced course segment.- 
This confirms the findings of Phases I and II. 

(2) In order to estimate completion time accurately in an Individualized multi- 
media instructional situation, the data needed to be grouped according to 
the instructional mode that was employed. This was shown to be the case 
when there were significant differences in the rate of learning by the media 
groups. Thus, media-specific predictive equations needed to be developed in 
those situations. This leads to the hypothesis that if tiie predictive 
instruments and the situation in which they are administered are more 
closely related to the instructional situation, predictions will be better. 
Further investigation of this hypothesis is needed. Evidence supporting it 
has been cited earlier (10, 15). 

(3) Although general aptitude measures (the ACBs) did not enter significantly 
into the "best*' predictive functions, the AFQT score, was a significant 
component of several useful equations. 

(4) The PREDICT computer program was developed for displaying the 
predictive inf oarmation. It was considered to be a useful aid to^ training 
managers of self-paced training programs. However, it is limited in that it 
requires access to a computer facility. 

(5) The use of nomographs to display predictive equations in a manner that 
can be used by training managers w^ successfully demonstrated. 
Nomographs are . simple to use and do not require access to a computer. 
They are recommended for use in the prediction system at QMS. 

(6) Although Instructional Time was adequately predicted by certain equations, 
test-taking time (as represented by the PEEs) was not. In fact, PEE time 
appeared to reduce the accuracy of predictions of Total Time, when thai 
score was the criterion. Further investigation of this matter is needed. 
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chapter 5 

PHASE IV . PROPOSED STUDY USING PREDICTIVE TEST BATTERY 3 



Currently, thc^ Annex D section of the 76P20 course is completely individualized. 
Instruction is presented in two modes (Pl-text and slide/tape). Annex D is a significant 
portion of the program (2/7 of the academic training time in the conventional course). 
The criterion time scores lo be employed in the next stage of the research effort will be, 
therefore, more indicative of the "ultimate" criterion— time in the total course. 

The first problem that should be studied is the feasibiiity of obtaining time scores as 
predictors in a pre-test situation. As the results "'of Phase III have shown, the most 
course-related predictors (earlier course segment times) were the "best" predictors of 
complet/on time of the individualized segment. Therefore, for the next test battery, time 
scores during the testing session should be obtained in addition to the accuracy scores 
(number correct) that were previously collected. It is expected that there may be 
administrative problems in reliably obtaining these scores. Trie students would be 
instructed to record their own starting and finishing times for each t^st in the battery. A 
digital wall clock would be provided, to enable the students to record the required times. 

Students would be told that the remaining time in the day would be free once they 
have completed the test battery. This should provide sufficient motivation for the student 
to work as efficiently and rapidly as possible. However, a situation that would not be 
reflected in the te.sting environment exists in the self-paced classroom. That is, in Ihe 
course, if a student fails a given module, he can be recycled or repeat the instruction for 
that module, and then take that test over again. In a pre-test situation, this would be 
impossible. Therefore, no detennination of a "failing" score for each test could be made 
in advance. All scores would be used in the development of the predictive function.s. 

The second major problem that should be studied is whether or not instructional 
situation-related scores arc more accurate predictors than those obtained in Phase III 
under fixed-time conditions. Forms would be prepared for use by QMS instructional 
personnel for recording performance and time scores in th^ individualized Annex D 
segment of the course. Each of the instructional modules withiji Annex D contains both 
an instructional and PEE component as did the B-3 segment studied in the previous 
Phase. Thus, there could be D-2, D-3, D-4, and D-5 Instructional Time criteria as well as 
t€st-taking time criteria for each ofHhese components. In addition, there is an Instruc- 
tional Time component for D-1, a CPE component (D-6), and a PEE component (D-7). 
In addition such scores as Total Time in Annex D, could be used as criterion variables for 
equation development and the determination of predictive accuracy. 

The predictors would consist of a refined and amended test battery, entry charac- 
teristic scores (AFQT, ACBs), and PEE scores and times in the first three course sections 
(Annexes A, B, and CJ). 

As a result o^ the anaiy.ses in Phase 111, Predictive Test Battery 3 was developed. 
Only those tests in Battery 2 that contributed significantly to the "best" predictive 
functions generated in Phase III were retained. In addition, four mini-lesson/test situations 
(PED-1, 2, 3, 4) were constructed to relaU^ specifically to the Annex D portion of the 
course. These instruments were developed from the course objectives in each of the four 
major instnictional modules in the Annex. A complete list of the tests to be used in 
Predictive Test Battery 3 is shown in Table 8. 
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Table 8 



Predictive Tes^ Battery 3 (Self-Paced)-Supply 
Course (76P20)\ Related, Refined and Amended 







Number of 


Test Type 


Test 


Items/Entries 


Arithmetic Skills and Knowledge 








Number Recognition 


24 




Symbol Recognition and Operation 


50 


Clerical Skills and Kn^ /ledge 








Number Comparison 


33 



Verbal Skills and Knowledge 



'Annex D Skills and Knowledges 
(Individualized portion of 
76P20 in Course) 



Affective 



Proofreading 

Vocabulary 
Reading Test-1 
Reading Test-2 
Reading Test-3 
Reading Test-6 
Reading Test-7 
Reading Test-10 
Practical Exercises-B 



Practical Exercise - D(1) 
Practical Exercise - D(2) 
Practical Exercise - 0(3) 
Practical Exercise - D(4) 

Supply Activity (Rating and 
Level of Aspiration) 



25 

39 
20 
23 

6 
17 

9 
12 
29 



83 
82 
77 
82 



15 



The major change in Battery 3 is the removal of time limits on each of the tests. 
This change requires an increased amount of written instruction. It also demands close 
monitoring of the students to ensure that they record all starting and finishing tim^s for 
every test. 

The same procedures used in Phase III for predictive equation development and 
evaluation would be employed in Phase IV. The ''best'' equations to be developed in^ this 
Phase would be compared for their accuracy with those developed in Phase III, to 
determine whether the additional administrative burden required to obtain time scores is 
**worth'' the effort. 

Other problems requiring study have been identified. Research is needed to 
determine the relationship between test-taking time and performance. Such data would 
indicate whether a linear prediction model is appropriate for use in predicting test-taking 
time. Recent studies (26) have questioned this assumption. Also, predictor instruments 
that validly reflect the AV instnjctional environment need to be developed. The student 
learns in a semi-enclosed carrel. He uses earphones attach»3d to a tape recorder, and views 
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a screen upon which shdes are projected. To produce an ^'instructional situation-related'' 
predictor, it is proposed that the students be permitted access to this environment during 
the testing session. Finally, a study needs to be performed to determine what kind of 
incentive system can be developed to enhance the predictive capability of the STOCK/ 
PRISM test battery. 

It is expected that approximately 200 students would be required for the proposed 
Phase IV research. By the end of FY 1973 the entire individualized 76P20 course is 
supposed to be completed. At that time, the utility and accuracy of the predictive 
functions and techniques generated by this research could be assessed. 
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Appendix A 



EXAMPLE OF STEPWISE MULTILE LINEAR 
REGRESSION ANALYSIS 

On the following pages are printouts of a stepwise multiple linear regression analysis 
that generated one of the predictive equations in Phase III. 

On printout A-1 the result of the first step in the analysis is shown. At this point 
the variable that had the highest correlation with the criterion was entered mto regres- 
sion. This was the A-5 PEE' time, which had a .688 correlation with the B-3 Instructional 
Time criterion. The regression coefficient is listed along with the / value, which repr(\sents 
a test of significance of the regression coefficient. 

On pri)itout A-2, tiie second variable {RT-3) enters the equation, RT-3 had the 
largest partial correlation coefficient w^ith the criterion of the variables remaining to be 
selected. That is, the second vaj'iable to enter was the one that added the most to the 
multiple correlation coefficient (in this case it raised it to .806). This yielded the best 
two-predictor equation from among the possible fimctions containing A-5 PEE time (tlie 
first variable that had been selected). Also presented in the [)rint()Ut are the beUi 
coefficients, which i)rovide ai^ indication of the relative importance of each jiredictive 
variable withiii the equation. In the case of the two predictors presented on printout A-2, 
their contributions are about ec|ual. 

On printout A-3, the third ^iU'p in the regression analysis is shown. Tin* B-2 PEE 
time was selected as the variable which, when combined with the first tv/o, produced th(^ 
best three-predictor equation. Also, the t values for each of the predictors were above 
1.96 (p = .95). This indicated that the multiple correlation coefficient was increased 
significantly at that level of confidence. In the same manner, the fourth v^iriable (AFQT) 
is shown on printout A-4 as significantly entering the equation. 

On the last printout (A-5), the fifth step in this process is displayed. RT 1 now 
enters the equation. However, the t value for RT 1 is below that selected as the cutoff 
(/ = 1.96) when visually insjx^cting the results of tiie analysis. A cutoff of f = 1.28 (/; = .80) 
was used ivithin the program to permit varial)!es, such as RT 1, to enter the equation. 
Such factors would not have done so with more stringent criteria. This approach was 
used to identify the viirial)les that almost "made it,'* and were still worth considering. A 
comparison between the ecjuation in printout A-4 and the one in printout A-5 shows that 
there was not much of an increas(^ in the muUiple correlation coefficient with the 
addition of the fifth predictor. Also, tlie standard en'or of estimate did not decrease 
greatly. A de(*ision was then made to use the ecjuation in printout A-T 



A-1 

STEPWISE MULTIPLE LINEAR REGRESSiOW 
PREDICTING B-3 INSTR TIME FOR CLASSES 101-127 (PI) 

STEP NUMBER 1 ENTER A-5 
STANDARD ERROR OF ESTIMATE= 92.228 
MULTIPLE CORRELATION COEFFICIENT = 0.688 
GOODNESS OF FIT,F( 1, 65)= 58.3261 
CONSTANT TERM= 100.2605 

VARIABLE NAME COEFF STD DEV T VALUE BETA COEFF 

COEFF 

A-5 7.0924 0.9287 7.6372 0.6877 



A-2 



STEPWISE MULTIPLE LINEAR RE 
PREDICTING B-3 INSTR TIME FOR CLA 



;SES 101-127 (PI) 



STEP NUMBER 2 ENTER RT 3 

STANDARD ERROR OF ESTIMATE= 75.847 
MULTIPLE CORRELATION COEFFICIENT = 0.806 
GOODNESS OF FIT,F( 2, 64) = 59.1735 
CONSTANT TERM= 314.7476 



VARIABLE NAME 

RT3 
A-5 



COEFF 

-40.5820 
5.3389 



STD DEV 
COEFF 
7.1619 
0.8240 



T VALUE 

-5.6663 
6.4790 



BETA COEFF 

-0.4528 
0.5177 



A-3 

STEPWISE MULTIPLE LINEAR REGRESSION 
PREDICTING B-3 INSTR TIME FOR CLASSES 101-127 (PI) 

STEP NUMBER 3 ENTER B-2 
STANDARD ERROR OF ESTIMATE= 70.956 
MULTIPLE CORRELATION COEFiFTCIENT = 0.835 
GOODNESS OF FIT,F( 3, 63)= 48.4518 
CONSTANT TERM= 236.0327 



VARIABLE NAME COEFF 

RT 3 -31.3937 

A-5 4.3227 

B-2 2.1444 



STD DEV 
COEFF 
7.2954 
0.8344 
0.6738 



T VALUE 

-4.3035 
5.1806 
3.1825 



BETA COEFF 

-0.3503 
0.4192 
0.2764 




A -4 

STEPWISE MULTIPLE LINEAR REGRESSION 
PREDICTING B-3 INSTR TIME FOR CLASSES 101-127 (PI) 

STEP NUMBER 4 ENTER AFQT 
STANDARD ERROR OF ESTIMATE= 68.058 
MULTIPLE CORRELATION COEFFICIENT = 0.852 
GOODNESS OF FIT, F( 4, 62)= 41.1192 
CONSTANT TERM= 338.007.3 



VARIABLE NAME COEFF 

AFQT -1.2510 

RT 3 -24.4079 

A-5 4.0903 

B-2 1.7803 



STD DEV 
COEFF 
0.4915 
7.5167 
0.8055 
0.6619 



T VALUE 

-2.5455 
-3.2472 
5.0778 
2.6896 



BETA COEFF 

-0.2085 

-0.2723 
0.3966 
0.2295 



A-5 

STEPWISE MULTIPLE LINEAR REGRESSION 
PREDICTING B-3 INSTR TIME FOR CLASSES 101-127 (PI) 

STEP NUMBER 5 ENTER RT 1 

STANDARD ERROR OF ESTIMATE-- 67.276 
MULTIPLE CORRELATION COEFFICIENT = 0.858 
GOODNESS OF FIT,F( 5, 61)=; 34.1539 
CONSTANT TERM= 382.1501 

VARIABLE NAME COEFF STD DEV T VALUE BETA COEFF 



. COEFF 

AFQT -1.3206 0.4878 -2.7071 -0.2201 

RTl -3.6348 2.3228 -1.5649 -0.1285 

RT3 ■ -18.2462 8.4092 --2. 1698 -0.2036 

A-5 3,7209 0.8305 4.4801 0.3608 

B-2 1.8661 0.6566 2.8420 0.2405 
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